Effects of Time Delay in Feedback Control of Linear Quantum Systems 
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We investigate feedback control of linear quantum systems subject to feedback-loop time delays. 
In particular, we examine the relation between the potentially achievable control performance and 
the time delays, and provide theoretical guidelines for the future experimental setup in two physical 
systems, which are typical in this research field. The evaluation criterion for the analysis is given 
by the optimal control performance formula, the derivation of which is from the classical control 
theoretic results about the input-output delay systems. 
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I. INTRODUCTION 

For reliable realization of quantum feedback control, 
it is indispensable to take into consideration some real- 
world limitations, such as incomplete knowledge of the 
physical systems and poor performance of the control 
devices. Various efforts on these issues have been un- 
dertaken in these few years, see e.g., [H, 0, 0, 0] for the 
system parameter uncertainty. Among such limitations, 
time delays in the feedback loop, which happen due to the 
finite computational speed of classical controller devices, 
are extremely serious, since their effect may completely 
lose the benefit of feedback control [!, fl 0] • To avoid the 
time delays, one can think to use the Markovian feedback 
control, in which the measurement results are directly fed 
back [8, 9]. However, while these experimental simplifi- 
cation has been extensively studied, theoretical ways to 
evaluate the effect of the time delays have not been pro- 
posed so far. 

In this paper, we investigate the effect of the time de- 
lays on the control performance, which is defined in terms 
of the cost function optimized by feedback control. This 
investigation provides theoretical guidelines for the feed- 
back control experiment. As the controlled object, the 
linear quantum systems are considered. In order to pre- 
pare the tool for the analysis, we first consider the opti- 
mal LQG control problem subject to the constant time 
delay. The optimal controller is obtained via the existing 
results in the classical control theory [icj |. Further, these 
results allow us to obtain the formula for the optimal 
value of the cost. 

The obtained formula enables us to examine the re- 
lation between the optimal control performance and the 
time delay both in an analytical and a numerical ways. 
Then, the intrinsic stability of the systems is dominant 
for the performance degradation effect. If the system is 
stable, the degradation effect converges to some value in 
the large time delay limit. Otherwise, the performance 
monotonically deteriorates as the delay length becomes 
larger. Based on this fact, we perform the analysis stated 
above for several physical systems that possess different 
stability properties. In addition to the controller design, 
we examine the relationship between the measurement 



apparatus and the best achievable performance. Based 
on this, we propose a detector parameter tuning policy 
for feedback control of the time-delayed systems. 

This paper is organized as follows. Linear quantum 
control systems are introduced in the next section. In 
Section III, we state the control problem for dealing with 
the time delay issue, and provide its optimal solution. 
In Section IV, we investigate the effect of the time delay 
in quantum feedback control based on two typical exam- 
ples possessing different stability properties. Section V 
concludes the paper. 

We use the following notation. For a matrix A = (fly), 
A T , A* and A* are defined by A T = (aji), A^ = (a* A 
and A* = (a*j), respectively, where the matrix element 
dij may be an operator and a*j denotes its adjoint. The 
symbols Re(A) and lm(A) denote the real and imaginary 
parts of A, respectively, i.e., Re(^4) = (A + A*)/2 and 
lm(A) — (A — A*)/2i. All the rules above are applied to 
any rectangular matrix. 



II. LINEAR QUANTUM SYSTEM 

Consider a quantum system which interacts with a vac- 
uum electromagnetic field through the system operator 



Cx, 



(1) 



where x — [g,p] T and C = [ci,C2] S C lx2 . When the 
system Hamiltonian is denoted by H , this interaction is 
described by a unitary operator Ut obeying the follow- 
ing quantum stochastic differential equation called the 
Hudson-Parthasarathy equation [TTI ] : 



dU t 



-iH 



1 



e'e 



dt + cdB 



c f dB f 



Ut, (2) 



where Uq is the identity operator. The field operators 
B\ and B t are the creation and annihilation operator 
processes, which satisfy the following quantum Ito rule: 

dB t dBj = dt, dB t dB t = dB\dB t = dBjdBj = 0. (3) 

Further, suppose that the system is trapped in a har- 
monic potential, and that a linear potential is an input 



2 



to the system. The system Hamiltonian H t at time t is 
given by 



H t = -x T Gx - x T T,Bu t 



(4) 



where u± £ 1 is the control input at time i, the sys- 
tem parameters G G R 2x2 and BeR 2 are a symmetric 
matrix and a column vector, and £ is given by 

1 

-1 

Then, by defining x t = [q t ,Pt] T = [U t qU} ,U t pU}] T and 
by using the commutation relation [q,p] = i and the 
quantum Ito formula, we obtain the following linear equa- 
tion: 

dx t = Ax t dt + Bu t dt + iY,{C J dB\ - CUB t ), (5) 

where A :— E[G + Im(C^C)]. Measurement processes 
are described as follows. Suppose that the field observ- 
able e^'^Bt+e^Bl is measured by the perfect homodyne 
detector, where <fi £ [0, 2tt) denotes the detector param- 
eter that the experimenter can change [T3|. Then, the 
output signal y t is obtained by 



Vi 



= Ul{e-^B t +e^B\)U t . 



(6) 



The simple calculation yields the infinitesimal increment 
of the observable yt as follows: 



d Vt = (e'^C + e t<p C*)x t dt + e~^dB t + e^dBj. (7) 

In the following section, we refer to (JSJ) and as the 
system dynamics and the output equation, respectively. 



III. OPTIMAL FEEDBACK CONTROL 
A. Input-output delay system 

As stated in the introduction, the effect of time delays 
is significant in feedback control of quantum systems. 
Those delays are mainly originated from the computa- 
tional time for a controller and the transition delay of 
signals. Thus, they should be modelled practically as 
input-output delays in the feedback loop, i.e., at time t, 
the signal u±-hi works as a control input for the system 
and the information {y s } s <t-h 2 is available in the con- 
troller, where we assume that hi and h 2 are constants. 
Without loss of generality, when we consider the opti- 
mal control problem for such a system, the total delay 
time can be simply put together into one input (or out- 
put) delay. Then, the system dynamics are modified as 
follows: 



dxt 



Ax t dt + But- h dt + iY,{C T dB\ - CUB t ). (8) 



Here, the real constant h denotes the total time delay in 
the feedback loop, i.e., h — hi + hi. Note here that u% 
should be determined by {y s } s <t- 



B. Optimal control performance 

We consider the optimal control problem for the sys- 
tem described by and ©. The following system ex- 
pression is convenient for exploiting results in the classi- 
cal control theory. Let us define a quantum noise vector 



er^Bt - 
-iB t 



ml 



(9) 



It is shown that the quantum noise vector satisfies the 
following properties: 



(w t ) = 0, 
dwtdwj = 



F^dt, if s = t 
0, otherwise 



(10) 
(11) 



where (•) denotes the expectation and F is the non- 
negative Hermitian matrix given by 



Fa, 



1 ie- % 



-re 



Also, we define the matrix S,p := ^(F^ +Fj). By substi- 
tuting the terms of the field observables B t , B\ with the 
noise vector wt, we obtain the following equations: 



dx t = Ax t dt + Bidw t + B 2 ut-hdt, 

z t = C\xt + DuUf-h, 
dyt = C 2 x t dt + D 2 \dw t . 



(12) 



Here, z% is an additional output signal defined to evaluate 
the system performance, and G\ € M 2x2 and D±2 E K 2 
are matrices freely tunable in controller design. The 
other system matrices are defined as follows: 



Bi 

B 2 
C 2 
D 2 i 



= £ Im 

= B, 

= er^C- 
= [l 0] 



2i 



2 exp(— i<f>) 
1 + exp(-i2</>) 1 + exp(-i2(f>) 



As depicted in Fig. [2 we investigate the feedback loop 
consisting of the system and a controller implemented 
by classical devices, such as analogue or digital circuits. 
Then, the optimal control problem is stated as follows. 



Problem 1 For the linear quantum system (|T2]) . find 
the causal, linear and time-invariant control law u : 
{lls}s<t — * u t that minimizes the cost functional 



J := lim (zjzt), 

t — >oo 

and determine the minimum value of J. 



(13) 



We make the following assumption, which is standard in 
the classical control theory; see [13[ for the details. 

Assumption 1 
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FIG. 1: Control of quantum systems by classical controllers 



1. (A, B 2 ) is stabilizable and (A, C 2 ) is detectable. 

2. For any (eM, 



A -XI B 2 
Ci D 12 



A Bt 
C 2 D21 



are row- and column-full rank, respectively. 

Ex := 
gular 



3. E x := D T 12 D 12 and E 2 ^ := D^S^D^ 



are nonsm- 



As in the other results for linear quantum systems, 
the solution of Problem [1] can be obtained by slightly 
modifying the derivation of the classical result in [iq |. 
Here, we only provide the minimum value of the cost J, 
which is of importance for the later discussion. For the 
specific form of the optimal controller, see the appendix. 



Theorem 1 Consider Problem Q] with Assumption Q] 
Let X, Y be the solutions of the matrix Riccati equa- 
tions XA + A T X + CjCx - F r ExF = and YA T + AY + 
BxS^Bj-LE^L 7 = with F := 1 (BjX+Dj 2 C 1 ) 
and L := -{YCj + BiS^Dj^E'l such that A + B 2 F, 
A + LC 2 are stable. Then, the optimal value of the cost 
functional J is given by 



Jo 



(14) 



where J° pt := tr^BxS^BjX) + tr(F T E 1 FY) is the opti- 
mal value of J when h = 0. 

Note that the existence of the Riccati solutions X and Y 
follows from Assumption [TJ 



IV. EFFECT OF FEEDBACK DELAY 

In the experiment of the feedback control, it is of im- 
portance to reduce the time delay by carefully setting 
up the experimental devices and achieve the best per- 
formance possible [f| 0|- However, some quantity of the 



time delay remains in practice. In this section, we inves- 
tigate how the time delay deteriorates the optimal con- 
trol performance by using the formula (fT4"| . In addition 
to the algorithm in the controller, we have the tunable 
parameter in the measurement apparatus. Thus, we do 
the analysis taking the detector parameter tuning into 
consideration. It should be noted that the optimal mea- 
surement technique was first introduced by Wiseman and 
Doherty Their technique is only for the delay-free 
systems, i.e., the optimization of the value of J^ pt . 

First of all, notice that the performance degradation 
effect is mainly determined by the exponential term in 
dH]). This means that the degradation is largely related 
to the system's intrinsic stability, i.e., the eigenvalues of 
the matrix A. Thus, it is obvious from the exponential 
growth of J^ P J that the unstable system easily deterio- 
rates as the time delay length increases, and that their 
control is significantly difficult. 

On the other hand, the remaining two classes, i.e., sta- 
ble and marginally stable systems, are relatively insen- 
sitive to the time delay and it is worth to analyze them 
in detail. In order to provide some guidelines for the 
experiments, we analyze the two physical systems that 
frequently appear in the context of quantum feedback 
control. In the following, we choose the matrices C\ and 
D 12 as 



Ci 



1 
1 



D 12 



Stable system - Consider a damped cavity with an 
on-threshold parametric down converter. The system 
Hamiltonian and the coupling operator are given by 

Ht = -^{qp + pq) - ut-hq, c = 8(q + ip), (15) 



where 7 > and 5 > are constant parameters. If they 
satisfy 7 < S 2 , the system is stable. In this case, clearly, 
J^ P J converges as h — > 00 since the real part of every 
eigenvalue of A is negative. When we choose the param- 
eter as 7 = 1/2 and 6=1, the optimal performance 
curves with the different detector parameters 4> & re given 
by Fig. [2 It is certainly confirmed that the performance 
degradation converges in the limit of h — > 00 for any de- 
tector parameters. The dashed line depicts the value of 
J when there is no control input field, i.e., u t = for 
any t > 0. We can see from the figure that even in the 
large delay limit, the appropriate measurement strategy 
significantly enhances the control performance compared 
to the uncontrolled case. 

In order to examine the effective detector parameter 
tuning, let us look at Figs. [2]and[3l These show that the 
optimal detector parameter hardly fluctuates over every 
delay length. In fact, we can see from Fig. [2] that the 
difference between two performance curves around the 
optimal one, i.e., the curve corresponding to <p = 1.98, is 
very small. Further, it is shown that the optimal mea- 
surement strategy in the delay-free case is perfectly the 
same as that in the large delay case. 



4 



Theorem 2 Consider Problem [T] with H t and c defined 
by (fT5|) . Let 0^ pt denote the detector parameter that 
minimizes J^ p ^ for the fixed delay length h. Then, the 
following relation holds: 



lim^r=^o Pt - 

n— s-oo 



(16) 



Proof. The goal is to show that the minimal val- 
ues of J^ pt and i are achieved with the same de- 
tector parameter value. Firstly, we compute the value of 
9J^ pt /90. Note that 

j; pt = tr(B 1 S^BjX)+t r (F T E 1 FY) 

= 5 2 (xn + x 22 ) + \{vi2 + l) 2 J/n 

+ (X12 + 1)(^22 + 1)2/12 + ^( X 22 + l) 2 ?/22, 

where 



X = 


3fXl 


2^12 


, Y = 


yn 


2/12 




_ 3512 


3522 _ 




V12 


2/22 



Then, since X is independent of 0, we obtain 



+ (X12 + 1) (3522 + 1) 



%L2 

90 



+ 2 ( - T22 + 1) 
Besides, the relation 

-^-(YA T + AY + BxSfiBl - LE 2 . 4> L T ) = 



provides us with 



(17) 



9 



['\Fe AT L) 2 dT 
Jo 



H Fe Ar (jL LL T) e A-T F T dT 



Fe J 



rfdY AT , A dY 



=—A' +A^— ) e A T F T dr 



1 f (35x2 + I) 2 dyn i\ d Vi2 
2\^^^ + (a;i2 + 1)(X22 + 1) ^ 

(x 22 + l) 2 dy 22 \ (x 12 + l) 2 2(S 2_ y)h dy n 



(X12 + 1)(3522 + 1) - 2 S 2 h f %12 

2 e \d(j) 

{x 22 + I) 2 2(S 2 +r)h ( dy^ 
4 I 90 



where we used 

A 

90 



(BiS^Bj) — 0, LE 2 ,(i>L T — LL T . 



1.36 



TOpt 

J fc.0 1.32 




FIG. 2: Optimal performance curves for a damped cavity 
with an on-threshold parametric down converter. A dashed 
line depicts the value of J when ut = 0. Three solid lines 
correspond to the optimal control performances with the de- 
tector parameters <f> = 1.68, 2.28, 1.98, from the top down- 
wards, respectively, where <j> = 1.98 is the optimal detector 
parameter. 



Thus, with the attention to (jTTJ), the following equation 
is obtained: 



■ iF ' ArLfdr 

1 8 J T , ( a; X2 + I) 2 - 2 (S 2 -r)h ( dyil 

2 90 4 V d( t> 

, ( X 12 + 1)(3522 +j_) e - 2( 5 2 h ^ %X2 



2 V 9 

(3^22 + i) 2 „- 2 (5 2 +r )h ( dyi 



122 



Hence, we obtain 
9Jl pt , 



lim 

h— *oo 



9j; 



opt 



90 



i^r 

2 90 1 



where the first equality follows from the pointwise conver- 
gence of and the uniform convergence of 9,7^/90 
on the domain of 0. This completes the proof. ■ 
From the discussion above, we can conclude that Wise- 
man's measurement strategy (the optimal tuning for 
delay-free case) is valid for the stable delay systems in 
that 0^ pt « 0g pt for any h > 0. 

Marginally stable system - The next system is a 
single particle trapped in the harmonic potential and cou- 
pled to the probe field via the position operator. The 
system Hamiltonian and the coupling operator are given 
by 



TJ 1 2 2 1 1 2 



ut-hq, c 



(18) 



where m and u are the mass of the particle and the an- 
gular frequency of the harmonic potential, respectively. 
For this system, the shape of the optimal performance 
curves is analytically calculated. 



5 
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FIG. 3: Optimal homodyne detector parameter v.s. each de- 
lay time for a damped cavity with an on-threshold parametric 
down converter. 



TOpt 

h,tf> 




FIG. 4: Optimal performance curves for a harmonic oscillator. 
A solid line, dashed line and chain line correspond to the 
optimal performances with <f) — 0, 27r/18, 37r/18, respectively. 



Theorem 3 Consider Problem [I] with H t and c defined 
by (|18[). Then there exist constants A, B and 9 such that 
the best achievable performance is given by 



ropt 

u h,opt 



J£ pt +A/i+ Bsin(w/i + 0). (19) 
Moreover, A and B are independent of the choice of <fi. 

Proof. Notice that the Riccati solution Y is dependent 
on the parameter </>. To make this dependence explicit, 
we write Y as Y<f, and, similarly, L as throughout the 
proof. Then, by Theorem [TJ the best achievable perfor- 
mance is given by 



Jo 



(20) 



where J^ pt is the positive constant given by 

J° pt = txiBxS^BjX) + tr(F T EiFYff,} 
On the other hand, direct computation yields 
Fe Ar L 6 



\ 



{f? + (mujf2) 2 }sm(uT + 6), 



where F — [fx, /a], L$ — [h,h] T and 9^ satisfies 
muj(fxh + hh) 



tan 6 = 



fxh - {moj) 2 f 2 h 



By combining this with (|20p . we obtain the first claim. 

It should be emphasized that (j> contributes to A and 
B only through 



V muj 



(21) 



Hence, it is sufficient to show the second claim that ([21 
does not depend on <j). When defining 



Y A = 



2/n 2/12 

2/12 2/22 



simple calculation yields 



(4 cos 2 cj))yl 2 + (2muj 2 - 4 sin 2^)y i2 + 4 sin 2 0-1 = 0, 



(2m cos 2 <j>)ylx - 2/12 = 0. 



Then, we obtain the following: 



mui 



2 2 4(cos0 J/12 - sin^) 2 

(4cos^ 4>)ytx + g"^ 

m to 

{ (4 cos 2 cf))y 2 2 + (2muj 2 ~ 4 sin 2<f))y 12 + 4 sin 2 



This completes the proof. ■ 

Roughly speaking, the first statement says that in- 
creases linearly with the oscillation as the delay length h 
becomes large. This is a natural result of the fact that the 
matrix A has only pure imaginary eigenvalues. On the 
other hand, the second statement gives us a nontrivial 
insight: the growth rate A and the oscillation amplitude 
B are independent of the detector parameter <j>. 

Hence, depending on the delay length, the importance 
of the choice of the measurement apparatus differs. If 
the delay length is small, the improvable performance 
level is sensitive to the value of h. On the other hand, 
if the system suffers from the large delay, the apparatus 
adjustment is not significant, since the improvable per- 
formance level is relatively small compared to the value 
of J^ P 0. Thus, if the time delay can be made sufficiently 
small, experimenters have to adjust the detector param- 
eter depending on the resulting delay length. For the 
illustration, see Fig 0] for m = u = 1, which illustrates 
the optimal performance curve 
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V. CONCLUSION 

In this paper, we investigated performance degrada- 
tion effects due to time delays in optimal control of lin- 
ear quantum systems. The analysis was performed by 
the optimal control performance formula from the clas- 
sical control theory. The obtained remarks are strongly 
related to the intrinsic stability of the physical systems. 
In particular, we performed intimate evaluations for two 
typical systems with different types of stability. These re- 
sults are expected to give useful guidelines for the future 
experiments. 

APPENDIX A: OPTIMAL FEEDBACK 
CONTROLLER FOR PROBLEM [T] 

Consider Problem [TJ With the same notation as that 
in Theorem [TJ the optimal feedback controller is given by 

dx t = (A + B 2 F + e Ah LC 2 e~ Ah )x t dt 

-e Ah L{dy t + TT t dt) (Al) 



u t = Fx t (A2) 
drjt = -C 2 e- Ah x t dt + (dy t + ir t dt), (A3) 



and the finite-time integration system 



n t = C 2 f e A ^- h - T ^B 2 u T dT. (A4) 

Jt-h 



It should be noted that the implementation of this con- 
troller involves infinite-dimensional elements. Thus, com- 
puters with the finite memory cannot implement it in a 
precise sense. Fortunately, however, it is known that the 
approximation method proposed in [l5j permits the con- 
trol with high accuracy. 
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